GH-48068: [C++][FlightRPC] Linux ODBC: Configure Dremio instance to allow remote testing by alinaliBQ · Pull Request #49695 · apache/arrow

alinaliBQ · 2026-04-08T21:40:33Z

Rationale for this change

The goal is to run remote tests against the Dremio instance on Ubuntu-latest runner, as Dremio docker instances are only available on Linux and they are not supported on macOS/Windows GitHub Actions.

What changes are included in this PR?

Added dremio to compose.yaml.
Added logic for setting up dremio account and creating a table in cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh.
Add ODBC remote test connection variable ARROW_FLIGHT_SQL_ODBC_CONN.
set up Dremio remote instance in CI

Note: This PR prepares the Dremio instance for remote testing, which is activated using ARROW_FLIGHT_SQL_ODBC_CONN={..., HOST=dremio.

Are these changes tested?

Tested in CI at local repository.

Are there any user-facing changes?

N/A

GitHub Issue: [C++] [FlightRPC][SQL][ODBC] Set Up Linux ODBC Driver Tests Against Dremio Docker #48068

github-actions · 2026-04-08T21:41:28Z

⚠️ GitHub issue #48068 has been automatically assigned in GitHub to PR creator.

alinaliBQ · 2026-04-09T16:24:11Z

@lidavidm @kou This PR is ready for review, please have a look

alinaliBQ

Addressed code review comments

alinaliBQ · 2026-04-22T17:10:19Z

@kou @lidavidm All comments are addressed, please have another look.
cc @justing-bq

justing-bq · 2026-05-21T20:50:37Z

@lidavidm @kou
Comments addressed. Please review.

Copilot

Pull request overview

This PR prepares the C++ Flight SQL ODBC CI environment to run remote tests against a Linux-only Dremio Docker instance by adding a Dremio service to the repo’s Docker Compose setup and automating initial Dremio bootstrap (user + test table) from CI.

Changes:

Added a Dremio bootstrap script to create an admin user, obtain an auth token, and create $scratch.ODBCTest.
Extended compose.yaml with a dremio service and injected an ODBC remote connection string into the ubuntu-cpp-odbc service.
Updated cpp_extra.yml to start Dremio and run the bootstrap script during the ODBC Linux job.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.

File	Description
`cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh`	New bootstrap script to provision Dremio for remote ODBC tests.
`compose.yaml`	Adds Dremio service + wires ODBC connection env var and dependency into the ODBC Docker job.
`.pre-commit-config.yaml`	Adds the new script to ShellCheck’s allowlist.
`.github/workflows/cpp_extra.yml`	Starts Dremio and runs the bootstrap script in the ODBC Linux workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

justing-bq · 2026-05-22T22:09:21Z

@kou @lidavidm
Comments addressed. Please review.

kou

Could you update the PR description?

Is this Dremio instance used in Linux tests? How to confirm it?

kou · 2026-04-30T11:33:53Z

+    environment:
+    - DREMIO_JAVA_SERVER_EXTRA_OPTS=-Dsaffron.default.charset=UTF-8 -Dsaffron.default.nationalcharset=UTF-8 -Dsaffron.default.collation.name=UTF-8$$en_US
+    healthcheck:
+      test: curl --fail http://localhost:9047 || exit 1


Why do we need || exit 1 here?

cc @justing-bq

I don't believe it is necessary so I removed it.

alinaliBQ · 2026-06-15T17:30:01Z

Could you update the PR description?

Is this Dremio instance used in Linux tests? How to confirm it?

I have updated the PR description to match the files updated.

The Dremio instance is used in the ODBC Linux tests through the environment variable ARROW_FLIGHT_SQL_ODBC_CONN. To confirm it, the ODBC test log should show FlightSQLODBCRemoteTestBase::* and FlightSQLOdbcV2RemoteTestBase::* tests.

These test suites exclusively connect to the remote instance indicated in ARROW_FLIGHT_SQL_ODBC_CONN, which in our case is the Dremio instance.

Authored by Justin Co-authored-by: justing-bq <justin.gossett@improving.com> ^ Conflicts: ^ .github/workflows/cpp_extra.yml

kou

+1

The Dremio instance is used in the ODBC Linux tests through the environment variable ARROW_FLIGHT_SQL_ODBC_CONN. To confirm it, the ODBC test log should show FlightSQLODBCRemoteTestBase::* and FlightSQLOdbcV2RemoteTestBase::* tests.

These test suites exclusively connect to the remote instance indicated in ARROW_FLIGHT_SQL_ODBC_CONN, which in our case is the Dremio instance.

Thanks. We can't confirm it in CI because our CI log doesn't show executed test names, right?

https://github.com/apache/arrow/actions/runs/27574204620/job/81518244076?pr=49695#step:7:10585

+ ctest --label-regex unittest --output-on-failure --parallel 4 --repeat until-pass:3 --timeout 300
Test project /build/cpp
      Start  1: arrow-array-test
      Start  2: arrow-buffer-test
      Start  3: arrow-extension-type-test
      Start  4: arrow-misc-test
 1/58 Test  #3: arrow-extension-type-test .............   Passed    0.09 sec
      Start  5: arrow-public-api-test
 2/58 Test  #2: arrow-buffer-test .....................   Passed    0.11 sec
      Start  6: arrow-scalar-test
 3/58 Test  #4: arrow-misc-test .......................   Passed    0.12 sec
      Start  7: arrow-type-test
 4/58 Test  #5: arrow-public-api-test .................   Passed    0.09 sec
      Start  8: arrow-table-test
 5/58 Test  #7: arrow-type-test .......................   Passed    0.11 sec
      Start  9: arrow-tensor-test
31/58 Test #34: arrow-compute-kernel-utility-test .....   Passed    0.08 sec
      Start 35: arrow-canonical-extensions-test
32/58 Test #35: arrow-canonical-extensions-test .......   Passed    0.09 sec
      Start 36: arrow-io-buffered-test
33/58 Test #33: arrow-compute-aggregate-test ..........   Passed    0.60 sec
      Start 37: arrow-io-compressed-test
34/58 Test #36: arrow-io-buffered-test ................   Passed    0.10 sec
      Start 38: arrow-io-file-test
35/58 Test #31: arrow-compute-vector-selection-test ...   Passed    1.00 sec
      Start 39: arrow-io-memory-test
36/58 Test #38: arrow-io-file-test ....................   Passed    0.16 sec
      Start 40: arrow-utility-test
37/58 Test #40: arrow-utility-test ....................   Passed    0.82 sec
      Start 41: arrow-async-utility-test
38/58 Test #37: arrow-io-compressed-test ..............   Passed    1.27 sec
      Start 42: arrow-bit-utility-test
39/58 Test #39: arrow-io-memory-test ..................   Passed    1.40 sec
      Start 43: arrow-crc32-test
40/58 Test #43: arrow-crc32-test ......................   Passed    0.14 sec
      Start 44: arrow-threading-utility-test
41/58 Test #42: arrow-bit-utility-test ................   Passed    1.85 sec
      Start 45: arrow-json-integration-test
42/58 Test #45: arrow-json-integration-test ...........   Passed    0.28 sec
      Start 46: arrow-filesystem-test
43/58 Test #46: arrow-filesystem-test .................   Passed    1.27 sec
      Start 47: arrow-flight-internals-test
44/58 Test #47: arrow-flight-internals-test ...........   Passed    0.07 sec
      Start 48: arrow-flight-test
45/58 Test  #1: arrow-array-test ......................   Passed    9.82 sec
      Start 49: arrow-flight-sql-test
46/58 Test #49: arrow-flight-sql-test .................   Passed    0.17 sec
      Start 50: arrow-odbc-spi-impl-test
47/58 Test #50: arrow-odbc-spi-impl-test ..............   Passed    0.08 sec
      Start 51: arrow-flight-sql-odbc-test
48/58 Test #48: arrow-flight-test .....................   Passed    1.26 sec
      Start 52: arrow-feather-test
49/58 Test #44: arrow-threading-utility-test ..........   Passed    4.60 sec
      Start 53: arrow-ipc-endianness-test
50/58 Test #53: arrow-ipc-endianness-test .............   Passed    0.15 sec
      Start 54: arrow-ipc-message-internal-test
51/58 Test #54: arrow-ipc-message-internal-test .......   Passed    0.11 sec
      Start 55: arrow-ipc-read-write-test
52/58 Test #52: arrow-feather-test ....................   Passed    0.47 sec
      Start 56: arrow-ipc-tensor-test
53/58 Test #56: arrow-ipc-tensor-test .................   Passed    0.10 sec
      Start 57: arrow-json-test
54/58 Test #57: arrow-json-test .......................   Passed    0.35 sec
      Start 58: arrow-telemetry-test
55/58 Test #41: arrow-async-utility-test ..............   Passed    6.00 sec
56/58 Test #58: arrow-telemetry-test ..................   Passed    0.11 sec
57/58 Test #55: arrow-ipc-read-write-test .............   Passed    2.16 sec
58/58 Test #51: arrow-flight-sql-odbc-test ............   Passed    9.48 sec

100% tests passed, 0 tests failed out of 58

conbench-apache-arrow · 2026-06-16T07:57:20Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit cb625c5.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.

alinaliBQ · 2026-06-16T17:28:47Z

Yes, that's right. Since enabling tests against the Dremio instance doubles the number of test cases being run, we can perhaps check by the amount of time it takes to run the arrow-flight-sql-odbc-test, which now takes longer as the ODBC driver establishes connection to the Dremio instance.

Before Dremio instance is enabled:
https://github.com/apache/arrow/actions/runs/27562488300/job/81477934843#step:6:15521

      Start 51: arrow-flight-sql-odbc-test
48/58 Test #51: arrow-flight-sql-odbc-test ............   Passed    0.29 sec

After Dremio instance is enabled shows ~9 sec.

github-actions Bot added Component: C++ Component: FlightRPC CI: Extra: C++ Run extra C++ CI awaiting review Awaiting review labels Apr 8, 2026

alinaliBQ marked this pull request as ready for review April 9, 2026 16:24

alinaliBQ requested review from assignUser, jonkeane, kou, lidavidm and raulcd as code owners April 9, 2026 16:24

lidavidm reviewed Apr 9, 2026

View reviewed changes

Comment thread cpp/src/arrow/flight/sql/odbc/tests/dremio/docker-compose.yml

github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Apr 9, 2026

kou reviewed Apr 10, 2026

View reviewed changes

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Apr 10, 2026

alinaliBQ force-pushed the gh-48068-dremio-docker-test branch 3 times, most recently from 7b6d340 to 4a892cf Compare April 10, 2026 21:20

alinaliBQ commented Apr 10, 2026

View reviewed changes

alinaliBQ requested review from kou and lidavidm April 17, 2026 17:08

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 4a892cf to 3cf2376 Compare April 22, 2026 17:18

kou reviewed Apr 27, 2026

View reviewed changes

Comment thread compose.yaml Outdated

Comment thread compose.yaml Outdated

Comment thread compose.yaml Outdated

Comment thread cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance.sh

github-actions Bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Apr 27, 2026

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 3cf2376 to 3d18ebd Compare April 27, 2026 23:44

justing-bq force-pushed the gh-48068-dremio-docker-test branch 2 times, most recently from 9ca9402 to fa7ab48 Compare May 21, 2026 16:34

kou requested a review from Copilot May 22, 2026 02:56

Copilot started reviewing on behalf of kou May 22, 2026 02:56 View session

Copilot AI reviewed May 22, 2026

View reviewed changes

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 70abfae to 35c324f Compare May 22, 2026 19:39

alinaliBQ mentioned this pull request May 25, 2026

GH-49482: [C++][FlightRPC][ODBC] Fix inconsistent SQLGetInfo values in global connection #50021

Open

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 35c324f to ad2866f Compare May 25, 2026 21:51

justing-bq force-pushed the gh-48068-dremio-docker-test branch 2 times, most recently from 91b8fa6 to 01d5339 Compare June 11, 2026 17:31

alinaliBQ requested a review from kou June 11, 2026 17:32

justing-bq force-pushed the gh-48068-dremio-docker-test branch from 01d5339 to 1e29550 Compare June 12, 2026 16:18

kou reviewed Jun 14, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Jun 14, 2026

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Jun 15, 2026

alinaliBQ and others added 2 commits June 15, 2026 13:26

Configure Dremio instance to allow remote testing

f9d0e36

Authored by Justin Co-authored-by: justing-bq <justin.gossett@improving.com> ^ Conflicts: ^ .github/workflows/cpp_extra.yml

Address review feedback

cc3b4fc

justing-bq force-pushed the gh-48068-dremio-docker-test branch from e6605f1 to cc3b4fc Compare June 15, 2026 20:26

kou approved these changes Jun 16, 2026

View reviewed changes

kou merged commit cb625c5 into apache:main Jun 16, 2026
62 of 64 checks passed

kou removed the awaiting change review Awaiting change review label Jun 16, 2026

kou mentioned this pull request Jun 16, 2026

[C++] [FlightRPC][SQL][ODBC] Set Up Linux ODBC Driver Tests Against Dremio Docker #48068

Closed

github-actions Bot added the awaiting merge Awaiting merge label Jun 16, 2026

Uh oh!

Conversation

alinaliBQ commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions Bot commented Apr 8, 2026

Uh oh!

alinaliBQ commented Apr 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alinaliBQ left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alinaliBQ commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

justing-bq commented May 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

justing-bq commented May 22, 2026

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

kou Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

alinaliBQ Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

justing-bq Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

alinaliBQ commented Jun 15, 2026

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

conbench-apache-arrow Bot commented Jun 16, 2026

Uh oh!

alinaliBQ commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

alinaliBQ commented Apr 8, 2026 •

edited

Loading

alinaliBQ commented Apr 22, 2026 •

edited

Loading

alinaliBQ commented Jun 16, 2026 •

edited

Loading